Integration and multidimensional sampling:

from the Gaussian integral to the Maxwell-Boltzmann distribution

In this lecture, we relate the concepts of sampling and of integration. The connection between the two was not clearly brought out before. We then treat general methods for directly sampling non-uniform distributions. A number of very detailed texts have been written on the subject, see for example.^[1]

Sample transformation: The example of Gaussian random numbers

Going up and down in the Gaussian integral

As an illustration of how sampling relates to integration, we consider the Gaussian integral:

$\int_{-\infty}^{\infty} \frac{dx}{\sqrt{2 \pi}} \exp(-x^2/2) = 1$

As we learned many years ago, we square this integral, write it down once in x and once in y:

$\begin{align} \left[\int_{-\infty}^{\infty} \frac{dx}{\sqrt{2 \pi}} \exp(-x^2/2) \right]^2 & = \int_{-\infty}^{\infty} \frac{dx}{\sqrt{2 \pi}} \exp(-x^2/2) \int_{-\infty}^{\infty} \frac{dy}{\sqrt{2 \pi}} \exp(-y^2/2)\\ \ldots & = \int_{-\infty}^{\infty} \frac{dx dy}{2 \pi} \exp[-(x^2+y^2)/2], \end{align}$

and then switch to polar coordinates
$dx dy = r dr d\phi$
to reach the following expression for the integral

$\begin{align} \ldots & = \int_{0}^{2 \pi} \frac{d\phi}{2 \pi} \int_{0}^{\infty} r dr \exp(-r^2/2), \end{align}$
We may then perform the substitutions
$r^2/2=\Upsilon (r dr = d\Upsilon)\quad \text{and}\quad \exp(-\Upsilon) = \Psi$
to reach

$\begin{align} \ldots & = \int_{0}^{2 \pi} \frac{d\phi}{2 \pi}\ \int_{0}^{\infty} d\Upsilon \exp(-\Upsilon)\\ \ldots & = \int_{0}^{2 \pi} \frac{d\phi}{2 \pi}\ \int_{0}^{1} d\Psi = 1. \end{align}$

In this last equation, we can compute the integrals (thus giving the value of the Gaussian integral), but we may in particular sample them:
Φ is a uniform random variable between 0 and 2π and ψ is a uniform random variable between 0 and 1. In the below program, we actually sample the random variables Φ and ψ, and transform these samples in several steps back to x and y. This procedure carries the name of "sample transformation".

Python program for the Gaussian sample transformation

Note that the substitutions are applied to the samples phi and Psi. This is also known as the Box-Muller transform.

from random import uniform
import math
import pylab
 
def gauss(sigma):
    phi = uniform(0.0, 2.0 * math.pi)
    Psi = uniform(0.0, 1.0)
    Upsilon = - math.log(Psi)
    r = sigma * math.sqrt(2.0 * Upsilon)
    x = r * math.cos(phi)
    y = r * math.sin(phi)
    return x, y
 
listx = []; listy = []
for k in range(10000):
    x, y = gauss(1.0)
    listx.append(x)
    listy.append(y)
list = listx + listy
pylab.figure(1)
pylab.hist(list, bins=25, normed=True)
pylab.title('Histogram Gaussian')
pylab.savefig('gaussians_histogram.png')
pylab.figure(2)
pylab.title('Correlation x,y')
pylab.axis('equal')
pylab.plot(listx,listy,'ro')
pylab.savefig('gaussians_correlations.png')
pylab.show()

Output of the above Python program

Here, we check that the correlations between x and y vanish.

Multi-dimensional Gaussians and the Maxwell distribution (theoretical treatment)

from mpl_toolkits.mplot3d import Axes3D
import pylab, random, math
 
ax = pylab.gca(projection='3d')
ax.set_aspect('equal')
x, y, z = [], [], []
for i in range(2000):
    a, b, c = random.gauss(0.0, 1.0), random.gauss(0.0, 1.0), random.gauss(0.0, 1.0)
    length = math.sqrt(a ** 2 + b ** 2 + c ** 2)
    x.append(a / length)
    y.append(b / length)
    z.append(c / length)
pylab.plot(x, y, z, '.')
pylab.savefig('surface_sphere.png')
pylab.show()
 

Output of the above Python program. Three rescaled Gaussians yield an isotropic distribution in space. Download the program to generate a figure that you can look at from different angles, using your computer mouse.

References

[∫∞−∞dx2π‾‾‾√exp(−x2/2)]2…=∫∞−∞dx2π‾‾‾√exp(−x2/2)∫∞−∞dy2π‾‾‾√exp(−y2/2)=∫∞−∞dxdy2πexp[−(x2+y2)/2],

---

1. ^ L. Devroye, Non-Uniform Random Variate Generation Springer 1986, p. 107 ff.cf http://cg.scs.carleton.ca/~luc/rnbookindex.html for the pdf of the entire book